There's Always an Exception

Whether we are learning to conjugate verbs in a new language or looking for patterns in how software is built, There's Always an Exception!

One of the most notable exceptions in the xUnit family relates to the use of a Testcase Object (page X) to represent each Test Method (page X) at runtime. This key design feature of xUnit is a way to achieve Independent Test (see Principles of Test Automation on page X). The only members of the family (that I am aware of) that don't do this are TestNG and NUnit (version 2.x). For reasons described below, the builders of NUnit 2.0 chose to stray from the well-worn path of one Testcase Object per Test Method and create only a single instance of the Testcase Class (page X) (which they call the test fixture) and reuse it for each Test Method. One of the authors of NUnit 2.0, James Newkirk writes:

I think one of the biggest screw-ups that was made when we wrote NUnit V2.0 was to not create a new instance of the test fixture class for each contained test method. I say "we" but I think this one was my fault. I did not quite understand the reasoning in JUnit for creating a new instance of the test fixture for each test method. I look back now and see that reusing the instance for each test method allows someone to store a member variable from one test and use it another. This can introduce execution order dependencies which for this type of testing is an anti-pattern. It is much better to fully isolate each test method from the other. This requires that a new object be created for each test method.

Unfortunately, this has some very interesting consequences when one is familiar with the "JUnit New Instance Behavior" of a separate Testcase Object per method. Since the object is reused, any objects it refers to via an instance variable is available to all the subsequent tests. This results in an implicit Shared Fixture (page X) along with all the various forms of Erratic Test (page X) that go with it. James goes on to say:

Since it would be difficult to change the way that NUnit works now, too many people would complain, I now make all of the member variables in test fixture classes static. It's almost like truth in advertising. The result is that there is only one instance of this variable no matter how many test fixture objects are created. If the variable is static then someone who may not be familiar with how NUnit executes would not assume that a new one is created before each test is executed. This is the closest I can get to how JUnit works without changing the way that NUnit executes test methods.

Martin Fowler felt this exception was important enough that he wrote an article about why JUnit's approach is right. See http://martinfowler.com/bliki/JunitNewInstance.html

Page generated at Wed Feb 09 16:39:03 +1100 2011